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Abstract 

Background: Realist synthesis is an increasingly popular approach to the review and synthesis of evidence, which 
focuses on understanding the mechanisms by which an intervention works (or not). There are few published 
examples of realist synthesis. This paper therefore fills a gap by describing, in detail, the process used for a realist 
review and synthesis to answer the question 'what interventions and strategies are effective in enabling 
evidence-informed healthcare?' The strengths and challenges of conducting realist review are also considered. 

Methods: The realist approach involves identifying underlying causal mechanisms and exploring how they work 
under what conditions. The stages of this review included: defining the scope of the review (concept mining and 
framework formulation); searching for and scrutinising the evidence; extracting and synthesising the evidence; and 
developing the narrative, including hypotheses. 

Results: Based on key terms and concepts related to various interventions to promote evidence-informed 
healthcare, we developed an outcome-focused theoretical framework. Questions were tailored for each of four 
theory/intervention areas within the theoretical framework and were used to guide development of a review and 
data extraction process. The search for literature within our first theory area, change agency, was executed and the 
screening procedure resulted in inclusion of 52 papers. Using the questions relevant to this theory area, data were 
extracted by one reviewer and validated by a second reviewer. Synthesis involved organisation of extracted data 
into evidence tables, theming and formulation of chains of inference, linking between the chains of inference, and 
hypothesis formulation. The narrative was developed around the hypotheses generated within the change agency 
theory area. 

Conclusions: Realist synthesis lends itself to the review of complex interventions because it accounts for context as 
well as outcomes in the process of systematically and transparently synthesising relevant literature. While realist 
synthesis demands flexible thinking and the ability to deal with complexity, the rewards include the potential for 
more pragmatic conclusions than alternative approaches to systematic reviewing. A separate publication will report 
the findings of the review. 



Background 

The range of approaches to research review and synthesis 
has been growing over recent years [1]. One approach that 
has been growing in popularity is realist synthesis. A real- 
ist review focuses on understanding and unpacking the 
mechanisms by which an intervention works (or fails to 
work), thereby providing an explanation, as opposed to a 
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judgment about how it works [2]. The realist approach is 
fundamentally concerned with theory development and 
refinement [2-4], accounting for context as well as out- 
comes in the process of systematically and transparently 
synthesizing relevant literature [3,4]. Given the complex, 
multifaceted nature of strategies and interventions used to 
promote evidence-informed healthcare, and the current 
limited understanding of their mechanisms of action, the 
realist approach is particularly suited to the synthesis of 
evidence about complex implementation interventions. 
Whilst the use of this method is increasing and the 
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evidence base growing [5-8], few published studies provide 
a detailed account of how it has been used. This paper 
therefore adds to the methodological evidence base about 
realist synthesis by describing application of the approach 
of an international project team (Realist Synthesis of 
Implementation Strategies (ReS-IS) to synthesise evidence 
about knowledge translation interventions for enabling 
evidence-informed healthcare. 

Realist synthesis 

Traditional systematic review approaches have been criti- 
cised for being too specific and inflexible [2,9,10]. This is 
an important consideration when examining the complex- 
ity of implementing health and social care interventions. 
The context of service delivery is complex, multi-faceted 
and dynamic, which arguably means that rarely would 
the same intervention work in the same way in different 
contexts. Consequently, conventional systematic review 
approaches to evaluating the evidence of whether inter- 
ventions work (or not) often result in limited answers such 
as 'to some extent' and sometimes' [4,9] . 

Realist review has emerged as a strategy for synthesising 
evidence and focuses on providing explanations for why 
interventions may or may not work, in what contexts, 
how and in what circumstances [11]. For example, Green- 
halgh et al [6] undertook a realist review to supplement a 
Cochrane review of school feeding programmes. Whilst 
the Cochrane review provided evidence that feeding pro- 
grammes work, it did not provide information about how 
they work and in what contexts. The findings from their 
realist review resulted in evidence regarding situations in 
which programmes may be more likely to be effective. 

The realist approach is philosophically rooted in real- 
ism, which combines three social science principles: 
causal explanations are achievable; social reality is mainly 
an interpretative reality of social actors; and social actors 
evaluate their social reality [12]. Realism involves identi- 
fying underlying causal mechanisms and exploring how 
they work under what conditions [13,14]. This context- 
ually bound approach to causality is represented as 
context + mechanism = outcome [15]. Therefore, it is an 
intuitively appealing approach to those trying to expose 
and unpack the complexities of contexts and interrelated 
mechanisms underlying implementation activity. 

The aim of a realist synthesis is . .to articulate under- 
lying programme theories and then to interrogate the 
existing evidence to find out whether and where these 
theories are pertinent and productive. Primary research 
is examined for its contribution to the developing theo- 
ry. . / [4]. In the context of implementation interven- 
tions, which are usually multi-faceted and complex, 
when setting out to develop and implement an interven- 
tion there is always an underlying theory about how it 
should work: if we do X in this way, then it will bring 



about an improved outcome [2,4,9]. The logic underpin- 
ning the aim of uncovering underlying theories about 
interventions is that no deterministic theories can always 
explain or predict outcomes in every context [16]. Fo- 
cussing on what it is about an intervention that makes it 
work (or not) in a given context should enable imple- 
mentation researchers to work at the level of mechan- 
isms of action. The premise is that in certain contexts 
individuals are likely (although not always certain) to 
make similar choices, and therefore particular contexts 
influence our choices such that reoccurring patterns 
emerge, Le., demi-regularities [4]. Realist review provides 
an approach to uncover the underlying theories that 
explain these demi-regularities by examining the interac- 
tions between mechanism, context, and outcome. 

A realist synthesis follows similar stages to a tra- 
ditional systematic review (Table 1), but with some nota- 
ble differences: 

1. The focus of the synthesis is derived from a 
negotiation between stakeholders and reviewers and 
therefore the extent of stakeholder involvement 
throughout the process is high. 

2. The search and appraisal of evidence is purposive and 
theoretically driven with the aim of refining theory. 

3. Multiple types of information and evidence can be 
included. 

4. The process is iterative. 

5. The findings from the synthesis focus on explaining 
to the reader why (or not) the intervention works 
and in what ways, to enable informed choices about 
further use and/or research [3]. 

The rest of this paper describes in detail the ReS-IS 
teams approach to applying the realist synthesis method 
to a review of interventions that enable evidence-informed 
healthcare, including the strengths and challenges encoun- 
tered in its use. A separate publication will describe the 
findings of the review and synthesis in detail. 

Methods 

Review purpose 

Numerous interventions have been applied and tested to 
promote the use of research evidence in practice. Imple- 
mentation of such interventions is often accompanied 
by complex strategies comprising support structures, 
resources and processes. While numerous systematic 
reviews have been conducted to determine the effective- 
ness of specific interventions [17-22], a systematic synthe- 
sis of the literature to examine the mechanisms by which 
such interventions work, and under what circumstances, 
has not been undertaken. Therefore the broad purpose 
of this review was to determine what interventions and 
strategies are effective in enabling evidence-informed 
healthcare. The specific purpose of the review was to 
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Table 1 Approach to realist review (adapted from Pawson [9]) 



Stage Action Activity 



Define the scope 
of the review 


Identify the question 


What is the nature and content of the intervention? 
What are the circumstances or context of its use? 
What are the policy intentions or objectives? 
What are the nature and form of its outcomes or impacts? 
Undertake exploratory searches to inform discussion 
with review stakeholders. 




Clarify the purpose(s) 
of the review 


Theory integrity - does the intervention work as predicted? 
Theory adjudication - which theories around the intervention 
seem to fit best? 

Comparison - how does the intervention 

work in different settings, for different groups? 

Reality testing - how does the policy intent of the intervention 

translate into practice? 




Find and articulate the 
programme theories 


Search for relevant 'theories' in the literature. 
Draw up list of programme theories. 
Group, categorise or synthesise theories. 

Design a theoretically based evaluative framework to be 'populated' with evidence. 
Develop bespoke data extraction forms. 


Search for and 
appraise the 
evidence 


Search for the evidence 


Decide and define purposive sampling strategy. 
Define search sources, terms and methods to be used 
(including cited reference searching). 
Set the thresholds for stopping searching at saturation. 




Test of relevance 


Test relevance - does the research address the theory 
under test? 

Test rigour - does the research support the conclusions 
drawn from it by the researchers or the reviewers? 


Extract and 
synthesise 
findings 


Extract the results 
Synthesise findings 


Extract data to populate the evaluative framework with evidence. 

Compare and contrast findings from different studies. 
Use findings from studies to address purposes(s) of review. 
Seek both confirmatory and contradictory findings. 
Refine programme theories in the light of evidence 
including findings from analysis of study data. 


Develop narrative 




Involve commissioners/decision makers in review of findings. 
Disseminate review with findings, conclusions and recommendations. 



establish what works, for whom, in what circumstances, 
and why with respect to interventions and strategies to 
enable evidence-informed healthcare. The purpose of the 
review was refined through stakeholder engagement at a 
knowledge utilisation colloquium meeting (http://www 
uofaweb.ualberta.ca/kusp/KUOXarchive.cfm). The stake- 
holders are a multi-disciplinary community of researchers, 
practitioners, and policy makers with expertise in know- 
ledge translation. This community served as a stakeholder 
reference group that were consulted at key stages in the 
review process: questions formulation, tool development, 
and evidence synthesis, providing critique and challenge 
to the method and emerging findings. 

Findings 

Defining the scope of the review: Concept mining and 
theory formulation 

This stage is fundamental to a realist synthesis because it 
provides the structure and framework for examining and 
synthesising diverse evidence [9]. The challenge of deve- 
loping a framework for a realist synthesis is in finding a 
level of abstraction that allows reviewers to stand back 



from the detail and variation in the evidence, but that is 
also specific enough to meet the purpose of the review. 
For a realist synthesis, an intervention is a theory; because 
interventions are implemented on a hypothesis of if we do 
X in this way, then it will bring about an outcome. This 
stage involves 'digging through' the literature and drawing 
on experience to identify key terms, concepts and mid- 
range theories that provide some explanation about the 
subject of interest [4]. The resultant model must be 
outcome-focused because a realist synthesis is con- 
cerned with uncovering what works' within differing 
contextual configurations. For this review, concept mi- 
ning and theory formulation was achieved through a mix- 
ture of face-to-face and virtual brain storming by the team 
familiar with the implementation and knowledge transla- 
tion research literature. 

A list of questions about interventions to promote 
knowledge use was developed and refined through exten- 
sive dialogue in small and larger group discussions. From 
these questions, the meaning of terms was clarified and 
concepts were identified. For example the meaning of 
terms such as evidence-based,' 'knowledge,' strategy,' and 
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'intervention' was discussed. It was agreed that an inter- 
vention was a broad concept, while strategies referred to 
mechanisms or approaches to achieve the intervention. 
Consensus was reached on the use of the term evidence- 
informed,' in preference to evidence-based,' on the 
grounds that decision making in practice, for good rea- 
sons, is not always research-based, but rather takes into 
account a range of factors in addition to empirical evi- 
dence [23]. We also developed a common understanding 
of terms related to realist synthesis (see Additional file 1). 

The group then identified common concepts among the 
questions developed in the previous activity. Eventually 
four key concepts, i.e., 'theory areas' emerged: change 
agency, technology, education and learning strategies, and 
systems change. These concepts were then framed to con- 
struct the theoretical framework, which underwent a 
number of iterations before declaration of the final 
framework. Figure 1 presents the final framework, 
which includes theory and contextual factors, dose and 
levels as central factors because of their importance 
in understanding implementation interventions, and 



outcomes is represented as an all-encompassing factor. 
The term 'dose' was used to describe how much of an 
intervention would bring about change. We recognise that 
the term dose does not necessarily fit well within the phi- 
losophy of realism, however in the context of this review 
we used it to ensure we paid attention to the relative 
strength of action of a mechanism within a particular con- 
text. The generic questions developed earlier were then 
customised to each of the four theory areas and these 
were used to guide the review and data extraction process 
(see Additional file 2 for foci and questions). 

The theoretical framework is made visible in a realist 
synthesis through the 'data extraction forms.' This is a 
unique feature of realist synthesis in that, unlike traditional 
systematic reviews, a bespoke set of data extraction forms 
are developed based on the content of the theoretical 
framework (see Additional file 3). The theory area ques- 
tions, described previously, provided the basis for develop- 
ment of these forms. In order to prevent misinterpretation 
of the data extraction questions, guidelines for using the 
forms were developed to explain each section and factors 



OUTCOMES 



1. Change Agency 

V 1 . What impact do the characteristics of 
the change agent have on evidence- 
informed healthcare? 

2. What is the overall impact of the 
change agent intervention on evidence- 
informed healthcare? 

3. What impact does the interaction 
between the change agent and the 
setting have on evidence-informed 
healthcare? 




2. Systems change 



Systems change interventions 
Interplay between setting and 
interventions 



DOSE 




■ 



LEVELS 



Theory & 
Contextual 
Factors 



v 



■ 

t 
# 
0 



3. Technology 



Effect of organizational 
characteristics on 
interventions used 
Effect of technological 
interventions 



4. Education 
and Learning 



Impact of education 
interventions 
Participatory versus non- 
participatory education 
interventions 



Figure 1 Review framework. 
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to consider when applying each question during extraction 
of information from an article. 

Pawson et al. [9] caution that completely comprehen- 
sive reviews may be impossible and recommend that the 
programme theories to be inspected be agreed upon and 
prioritised. As the framework reveals, the scope of our 
work was potentially vast. Therefore, the ReS-IS team 
engaged in discussion with the wider knowledge utilisation 
community through presentations and discussions, and as 
a result adopted a pragmatic approach that resulted in 
prioritisation of one theory area within the framework, 
change agency, for a first review. The programme theory 
questions for this theory area were: 

1. What impact do the characteristics of the change 
agent have on evidence- informed healthcare? 

2. What is the overall impact of the change agent 
intervention on evidence-informed healthcare? 

3. What impact does the interaction between the 
change agent and the setting have on evidence- 
informed healthcare? 

These questions were framed as what' questions in 
order to help us determine what it is about the particular 
actions (mechanisms) of change agents that have an 
impact and connect themes to action. Overall our 
approach to analysis was concerned with understanding 
why mechanisms were having an effect (or not). 

Search for, and appraisal of evidence 
Search approach 

In realist synthesis the literature needs to be scrutinised 
to identify studies related to the targeted programme 



theories' [9]. This focus does not mean that the approach 
to search and appraise literature is any less rigorous or 
systematic than approaches used in traditional reviews. 
The search is purposive in that for each of the theory areas 
the group embarked on producing a list of relevant and 
related search terms. The final list of terms, in conjunction 
with relevant indexing terms, was used to guide the 
searches, which were conducted by two team members in 
consultation with their institution s health science librarians 
(see Additional file 4 for search terms and strategy). 

Six online databases were searched: Medline, CINAHL, 
Embase, Psyclnfo, Sociological Abstracts, and Web of 
Science. Ovid was used to execute the search within a 
10-year publication period, which was considered an 
appropriate timeframe in the search for intervention studies. 
As a quality measure, one group member reviewed the 
indexes of 14 journals that publish articles about know- 
ledge utilization issues. A second group member deter- 
mined that relevant papers from these journals were 
adequately indexed in the databases selected. Addition- 
ally, using their knowledge of the literature all team 
members reviewed the final reference list to ensure 
there were no obvious omissions. Although Pawson 
et al. [9] describe the relevance of snowballing and con- 
sultation with experts as part of the realist review 
process, we did not have the resources to extend our 
search beyond the databases described above, and there- 
fore acknowledge we may have missed some key pieces 
of evidence. 

Search results were saved as text files and downloaded 
into Reference Manager Professional Version 11.0. The 
content of the file was then backed-up to a secure 



Search Strategy Development 
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server. Over 24,000 electronic references were returned 
from the change agents search strategies, which were 
screened through a process summarised in Figure 2. 

The preliminary screen was intentionally inclusive to 
capture all articles potentially relevant to the review 
purpose, including those outside of healthcare. We erred 
on the side of inclusion wherever a title appeared to be 
even potentially relevant to the change agent/agency 
concept or any of the terms of reference developed in 
relation to the change agency component of the theore- 
tical framework. At this stage, all seemingly relevant 
papers were retrieved in full-text for a more detailed rele- 
vance test: Is the evidence provided in this theory area 
good and relevant enough to be included in the synthesis/ 

Using criteria for exclusion, in the next stage we 
undertook a preliminary screening of the article titles 
retrieved in the search and reduced the list of potentially 
relevant papers to 196. Second-level screening resulted 
in the exclusion of a further 144 papers, bringing the 
final number of papers relevant to this theory area to 52. 
This process was managed by two team members, in 
consultation with the wider group as appropriate. Our 
consistent reference point was the review framework 
and related theory area questions. Therefore decisions 
about what was eventually included and excluded was 
managed through discussions about whether the papers 
were of direct relevance to interventions that had been 
evaluated within a healthcare context. 

The primary reasons for exclusion of papers at this 
level included: 

1. They were not research based (using the broadest 
definition of research, i.e., demonstrating a 
systematic approach to inquiry). 

2. They were purely an anecdotal account. 

3. They were not relevant to healthcare, e.g., were 
based on an unrelated field such as banking (we did 
not have resources to undertake a review of the 
wider literature). 

Appraisal 

To test the usability and functionality of the data extrac- 
tion form and to promote a consistent approach to data 
extraction, the tool was pre-tested, by all group members, 
on two purposefully selected articles. Pre-testing resulted 
in some minor modifications to the instrument prior to 
commencement of the review process. The group divided 
into five subgroups and the references related to the 
change agent theory area were divided evenly across the 
subgroups for appraisal. 

In contrast to other review processes, in a pure realist 
synthesis no literature is excluded (unless it does not relate 
to any of the theory areas). Because a paper is excluded in 
one theory area does not mean that it is necessarily 



excluded in other theory areas. Therefore, exclusion 
criteria at this level need to be explicit and a clear ration- 
ale needs to be documented for each article that is 
excluded. Consistent with Pawson's suggestions, in this re- 
view the test for inclusion was: Is the evidence provided in 
this theory area 'good and relevant enough' to be included 
(consider issues of sample size, data collection, data 
analysis, and claims made)? 

Discrepancies in opinions about the relevance of articles 
were resolved through discussion amongst the review 
group. Questions about relevance were guided by the 
theory area questions, and how particular pieces of 
evidence did or did not inform these. Final decisions erred 
on inclusiveness, rather than exclusion. 

Extraction and synthesis 
Data extraction 

If considered relevant, data were extracted from the article 
and then peer reviewed and checked by a second member 
of the subgroup. Even though the initial review wave 
comprised articles related to the change agent theory area, 
questions relating to each theory were applied during the 
extraction process because many of the articles reported 
aspects that were also relevant to the other theory areas. 
As the contents of the reviews theoretical framework were 
embedded in data extraction forms, these provided a 
template to 'interrogate' the papers. When extracting data, 
if an article did not include information relevant to a ques- 
tion in the form, the extractor recorded 'Not reported/ 
Direct quotations from the article were often most in- 
formative and were accompanied by the page number 
from which the quote was taken. Data pertaining to 
the target population and discipline for each study were 
also extracted. 

The aim of the data extraction process is to populate 
the evaluative framework with evidence. Therefore, once 
all subgroups had completed data extraction for the arti- 
cles deemed relevant to the review purpose, the content 
from each groups data extraction tables was amalga- 
mated to form a single data extraction table including all 
articles addressing change agency. 



Data synthesis 

The basic task of the synthesis process is to refine the 
programme theory; Le., to determine what works, for 
whom, in what circumstances, in what respects and why. 
Pawson et al! publications provide little guidance on how 
to approach data synthesis. They suggest that synthesis 
should focus on four dimensions: questioning the integrity 
of a theory, adjudicating between competing theories, 
considering the same theory in comparative settings, or 
comparing the 'official' theory with actual practice [4,9]. 
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In this review we developed an approach to synthesis 
based on the principles of realist evaluation [15], inclu- 
ding the following steps: 

1. Organisation of extracted data into evidence tables, 

2. Theming by individual reviewers, 

3. Comparison of reviewers' themes for a specific 
article and formulation of chains of inference from 
the identified themes, 

4. Linking of the chains of inference, and tracking and 
linking of articles 

5. Hypothesis formulation. 

This process is summarised in Additional file 5. 

At a face-to-face meeting, sub-groups undertook a 
process of analysis, focussed on addressing the three 
programme theory questions (see Figure 1 and 'Defining 
The Scope' section above). 

Each question was assigned to a subgroup, and its mem- 
bers' independently themed relevant data extracted from 
each article. The subgroup then collated the themes identi- 
fied by each of the members. At this point, emerging find- 
ings were challenged, and contrary examples sought. From 
here the subgroup members identified chains of inference. 
A chain of inference is a connection that can be made 
across articles based on the themes identified. Over a period 
of time, through virtual and face-to-face meetings, chains of 



interference were developed and refined. This process was 
both inductive and deductive. We did not get to the stage 
of retroduction within this review, which could therefore be 
viewed as a limitation of the review process reported here. 
Further work would need to be undertaken on the hypoth- 
eses to make better inferences about generative mechanisms. 

The group then formulated hypotheses regarding the 
chains of inference. Thus, themes were linked to chains of 
inference, which were then linked to a hypothesis. Further, 
all papers were explicitly linked to chains of inference and 
hypotheses. Additional file 5 includes full details of the 
stages of synthesis, and Additional file 6 presents full details 
of the hypotheses linked to themes, chains of inference, and 
papers. Tables 2 and 3 provide a summary of these stages. 

Development of narrative 

Typically the writing of the review follows the theoretical 
framework model developed. For this review, the narra- 
tive was organised according to hypotheses generated 
within the Change Agency theory area, with the data for 
each theme linked in two ways; to one another within 
each hypothesis, and also across hypotheses. Whilst the 
findings from this theory area will be presented in a sep- 
arate publication, the following presents a summary to 
illustrate our approach to analysis and synthesis. 



Table 2 Chains of inference linked to themes and original articles 



Chains of Inference 



Derived from the following themes 



Articles 



Knowledge 



Skills 



Personal Characteristics 



Social Interaction 



Professional qualifications 
Expert knowledge 
Knowledgeable 
Local knowledge 
Research Knowledge 
Practice knowledge 

Communication skills 
Leadership skills 
Thinking skills 
Clinical skills 
Cognitive skills 
Evaluation skills 
Political skills 
Facilitation Skills 
Reflective skills 

Role model 

Positive attitude 

Responsibility/accountability 

Respected 

Information Seeking 

Positive Attitude 

Accessible 

Age 

Teacher 

Culturally compatible 

Objectivity 

Years of experience 

Social Influence 
Networking 
Shared Ownership 



1, 3, 6, 7, 10, 11, 13, 14, 15, 16, 18, 19, 
20,21,22, 23, 25, 29, 35, 36, 37, 39 



2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,15,16, 17, 18, 19, 
20, 21, 22, 24, 25, 27, 28, 32, 33, 34, 36, 38, 39, 40 



1,2, 4, 6, 7, 8, 13, 14, 15, 16, 17, 18, 

22, 28, 29, 30, 31, 32, 33, 35, 36, 37, 38, 39 



5, 8, 12, 15, 18, 31, 39, 40, 
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Table 3 Hypotheses linked to chains of inference 



Hypotheses 



Chain of inference 
(theory level) 



Chain of inference 
(sub-theory level) 



Themes from 
the literature 



Papers addressing 
the theme 



An opinion leader and his/her 
personal characteristics are 
dependent on contextual 
factors in order to have an 
impact on E-IHC 
A facilitator and his/her personal 
characteristics are dependent on 
contextual factors in order to have 
an impact on E-IHC 



The nature of the 
relationship between 
the change agent's 
personal characteristics, 
the role adopted, 
and contextual influences 
and the impact of E-IHC 



Roles 

Personal Characteristics 
Contextual Factors 



Opinion Leader (OL) 
Facilitator (FAC) 
Change agent(CA) 



Papers with mixed and 
positive effects, only: 
6 OL (Wright, Chaillet, Curran, 
Moore, Davies, Majumdar) 
Six FAC (internal/external and 
external facilitators 
included), (Stetler, Cranney, 
Gerrish, Milner, Thomas, Hutt) 
Total 18 CA papers, 12 
OL and FAC 



Step one Data from extraction tables were summarised 
and organised into theory area and questions related 
tables, for example, in relation to the question what 
impact do the characteristics of the change agents have 
on knowledge utilization,' the table would include 
extracts of data, and the link back to the source papers. 

Step two These data were then themed, those themes 
were challenged and contrary evidence sought. In rela- 
tion to the characteristics of change agents a number of 
emerging issues emerged that could be considered as 
relevant conditions for change agency within an opinion 
leader role for example, such as confidence, years 
of experience, level of qualification (educated to post- 
graduate level), and willingness to work collaboratively. 

Step three Looking for chains of inference (connections) 
across extracted data and themes. Through an iterative 
process, connections were looked for across data/themes 
to build up a cumulative picture. For example, were the 
opinion leader conditions evident in papers about other 
change agent roles — this resulted in a larger list factors, 
including, for example attitude, expert knowledge, gen- 
der, age, and cultural compatibility (see Table 2 for more 
examples), leadership, being embedded, tailoring, part- 
nerships, influence, culture, support, and resources. 

Step four Hypotheses formation (mechanism, context, 
outcome chains). The output from steps three and four 
resulted in a cumulative picture of potential mecha- 
nisms, contexts and outcome chains (hypotheses), which 
could be linked back to source evidence (see Table 3 for 
an example). Our review indicates that, for example, 
change agents who are adequately supported and 
resourced (context) who role model the practices they 
espouse (mechanism) may impact more positively on 
achieving evidence-informed healthcare (outcome). 

The hypotheses acted as synthesised statements of 
findings against which the previous stages of analysis 
could be presented. The narrative was developed around 
each hypotheses, and summarised the nature of the 



context, mechanism and outcome links, and the charac- 
teristics of the evidence underpinning them 

Pawson et al. [9] do not suggest that recommenda- 
tions' per se are developed from realist synthesis, as 
the purpose is not to determine 'best' practice, but to 
describe the relationships between interventions and the 
contexts in which those interventions occur. They do 
however suggest that stakeholders are engaged both in 
the process of Validating' the emerging findings and in 
dissemination activities. To that end, our review 
approach and the developing findings were shared with 
a community of knowledge utilisation researchers and 
practitioners at an annual colloquium in 2009. This 
process helped to refine the focus and presentation of 
the narrative from the programme theory area. It also 
validated our view that some new insights about change 
agency had emerged from this review process, which will 
be reported in a separate paper. 

Discussion 

There are few published examples of realist syntheses 
and those that exist do not include a detailed account of 
the approach used because authors tend to focus on the 
dissemination of findings within publications [6,7,24-27]. 
This lack of information about application of the realist 
approach is unhelpful to a novice realist reviewer. To fill 
this gap, this paper presents in some detail the approach 
we took to conduct a realist synthesis of evidence about 
the effect of change agency on evidence-informed 
healthcare. Change agency is a complex implementation 
intervention, which made realist synthesis an appropri- 
ate approach for unpacking its effects within different 
contexts and groups. However, undertaking this review 
was not without its challenges, not least because of the 
practicalities of working as an unfunded and geographic- 
ally dispersed group. 

Our approach deviated in some respects from the 
approach recommended by Pawson et al. For example, 
Pawson et al. do not advocate a comprehensive litera- 
ture search, or double reviews and data extraction. In 
this sense we developed a hybrid approach that was 
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fundamentally rooted in realist synthesis philosophy and 
principles (i.e., theory led, purposive, iterative, stake- 
holder involvement), but which also drew on some of 
the practices of traditional systematic reviewing. 

One of the strengths of realist review is the approach's 
firm roots in philosophy and social sciences [2,9]. Rather 
than being a method or formula, it is a logic of enquiry' 
[9], which enables a flexible, all-embracing approach to 
explanation of what works for whom in what circum- 
stances and in what respects. Rather than controlling for 
real life events, realist synthesis provides a framework 
for working with and untangling the complexity of real- 
life implementation. This allows for an equal focus on 
what works, as much as what does not work, in an at- 
tempt to learn from failures and maximise learning 
across policy, disciplinary and organisational boundaries. 
Furthermore, realist synthesis is inherently stakeholder 
driven, which facilitates engagement and the inclusion of 
multiple perspectives. 

The strengths of realist synthesis underpin its limita- 
tions. Realist synthesis is premised on a set of principles 
rather than a formula, and whilst this allows for flexibi- 
lity and inclusivity, it means that the findings from a 
review are theoretically transferable. For example, it 
follows that if the appraisal and data extraction needs to 
be bespoke to the particular review questions that arise 
from the theoretical framework, these will be different 
for each review. Furthermore, given that the fundamen- 
tal interest in realist synthesis is about finding out what 
works in what contexts, the recommendations one can 
make will not be generalisable. Rather a realist review 
results in findings that are theoretically transferable; 
ideas ('theories') that can be tested in different contexts, 
with different stakeholders. 

Pawson et al. [2] suggest that realist syntheses are not 
for novices. Unlike a Cochrane review, for example, 
which relies on standardised protocols and tools, the 
demands on a realist synthesiser are different. For ex- 
ample, quality assurance within realist synthesis is 
dependent on reviewers' explicitness and reflexivity. 
During this review, we kept a log of the process and 
decisions made throughout the process, which we devel- 
oped into a technical report. In addition, we undertook a 
more formal reflective process during the review because 
members of the group had varying experiences of realist 
review. This involved reflecting on questions about what 
was going well, what was going less well, as well as en- 
gaging in group learning activities. Throughout the re- 
view process, we had large and small group discussions 
that provided the opportunity for building in checks and 
balances, and for explicating processes. In turn, this 
requires a high level of expertise in reasoning, research 
methods and quality appraisal, and expertise in the 
subject area. The complexity of the realist synthesis 



approach means that it is time-consuming and human 
resource intensive, and for those reasons a potentially 
expensive endeavour. 



Conclusions 

Realist synthesis is a new but emerging approach to 
evidence review. In this paper, we have described our use 
and development of the approach. It is particularly 
appropriate for unpacking the impact of complex inter- 
ventions because it works on the premise that one needs 
to understand how interventions work in different con- 
texts, and why. It is not an easy option. Realist review 
demands much of the reviewer, including an ability to 
think flexibly and deal with complexity. There is not one 
prescribed approach to doing a realist synthesis; rather, 
there is a set of principles that the reviewer must particu- 
larise to the issue being explored whilst being sympa- 
thetic to the philosophy of realism. This presents unique 
challenges, but with it, the opportunity to develop more 
pragmatically insightful conclusions than those produced 
by some other approaches to systematic reviewing. 
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